Privacy- Aware Mechanism Design 

Kobbi Nissim* Claudio Orlandi''' Rann Smorodinsky-t 

February 15, 2012 

Abstract 

Mechanism design deals with distributed algorithms that are executed with self-interested agents. The 
designer's, whose objective is to optimize some function of the agents private types, needs to construct 
a computation that takes into account agent incentives which are not necessarily in alignment with the 
objective of the mechanism. Traditionally, mechanisms are designed for agents who only care about 
the utility they derive from the mechanism outcome. This outcome often fully or partially discloses 
agent declare types. Such mechanisms may become inadequate when agents are privacy- aware, i.e., 
when their loss of privacy adversely affects their utility. In such cases ignoring privacy-awareness in the 
design of a mechanism may render it not incentive compatible, and hence inefhcient. Interestingly, and 
somewhat counter-intuitively, Xiao [eprint 2011] has recently showed that this can happen even when 
the mechanism preserves a strong notion of privacy. Towards constructing mechanisms for privacy-aware 
agents, we put forward and justify a model of privacy-aware mechanism design. We then show that 
privacy-aware mechanisms are feasible. The following is a summary of our contributions: 

• Modeling privacy-aware agents: We propose a new model of privacy- aware agents where agents 
need only have a conservative upper bound on how loss of privacy adversely affects their utility. 
This is in deviation from prior modeling which required full characterization. 

• Privacy of the privacy loss valuations: Privacy valuations are often sensitive on their own. 
Our model of privacy-aware mechanisms takes into account the loss of utility due to information 
leaked about these valuations. 

• Guarantees for agents with high privacy valuations: As it is impossible to guarantee in- 
centive compatibility for agents that have arbitrarily high privacy valuations, we require a privacy- 
aware mechanism to set a threshold such that the mechanism is incentive compatible w.r.t. agents 
whose privacy valuations are below the threshold, and differential privacy is guaranteed for all other 
agents. 

• Constructing privacy-aware mechanisms: We first construct a privacy-aware mechanism for 
a simple polling problem, and then give a more general result, based on recent generic construction 
of approximately additive mechanisms by Nissim, Smorodinsky, and Tennenholtz [ITCS 2012]. We 
show that under a mild assumption on the distribution of privacy valuations (namely, that valuations 
are bounded for all but a diminishing fraction of the population) these constructions are incentive 
compatible w.r.t. almost all agents, and hence give an approximation of the optimum. Finally, we 
show how to apply our generic construction to get a mechanism for privacy-aware selling of digital 
goods. 
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1 Introduction 



Mechanism design deals with distributed algorithms that are executed with self-motivated agents who opti- 
mize their own objective functions. The mechanism designer, interested in computing some function of the 
agents' private inputs (henceforth types), needs hence to construct a computation that takes into account 
the agents' incentives, which are not necessarily in alignment with the goals of the designer. Settings where 
mechanism design is instrumental include centralized allocation of resources, pricing, the level of provision of 
a public good, etc. Traditionally, agents are modeled to care about the utility they derive from the outcome 
of the mechanism, but not about their privacy. Consequently, in many cases the outcome of the mechanism 
fully discloses the types declared by (some or all) agents. 

Wc look at a model where agents also assign non-positive utility to the leakage of information about 
their private types through the public outcome of the mechanism. This modeling is rele vant, e .g., when 
private information is aggregated via markets which provide superior prediction power (e.g., (WZ04 | ), kidne y 



exchange markets where information aggregation and sharing lead to huge health-care benefits (e.g., jARll 



or recommendation engines which assist individuals in locating optimal products. Such markets may not be 
incentive compatible and consequently can fail if agents' privacy is not accounted for. 

Our work is on the interface of the research in Algorithmic Ga me The ory and the recent theoretical 
research of privacy. Earlier s cholarly w ork by McSherry and Talwar MT07l | has forged a link between the 
notion of differential privacy |DMNSOa | and mechanism design. They observed that differential privacy can 
serve as a tool for constructing mechanisms where truthful ness is e -dominant. A recent work |NST12 | has 



observed a few weaknesses in constructions resulting from MT07l | and resolved them by putting forward 
a general framework for constructing approximately-optimal mechanisms where truthfulness is a dominant 
strategy or an ex-post Nash equilibrium. This line of work demonstrates that differential privacy can serve 
as a powerful tooZ for the construc tion of efficient mechanisms. 

The mechanisms presented in |MT07 . NST12] were not analyzed with respect to agents who take into 
account their dis-utility due to the information leaked about their types. We call this dis-utility information 
utility and we call privacy-aware agents those agents that take the information utility into account. It might 
be tempting to think that the combination of truthfulness and differential privacy is always sufficient for 
making privacy-aware agents truthful - mechanisms that are truthful and preserve differential privacy should 
remain truthful also with respect to agents that take information utility into account. A work of Xiao (Xiallj 
dispels this intuition by showing a mechanism that preserves differential privacy and is truthful with respect 
to agents that are not privacy aware, yet, under what seems to be a reasonable definition of information 
utility, truthfulness is not dominant with res pect to privacy-aware agents. 

A recent work of Ghosh and Roth |GRll| constructs mechanisms that compensate agents for their loss in 
privacy. Ghosh and Roth consider a setting where a data analyst wishing to perform a differentially private 
computation of a statistic pays the participating agents for using their data. They construct mechanisms 
where agents declare how their loss of utility depends on the privacy parameter, and the mechanism decides 
upon which agents' information will b e used in the computation and how much they will be paid. Inter- 
estingly, the mechanisms presented in |GR11 | do not preserve the privacy of the loss valuations. However 
an agent value for privacy can reveal information about the agents' private data: it is not unreasonable to 
assume that there is some correlation between the price and agents sets on her privacy and the unlikelihood 
of her private data or, in other words, to assume that people value their privacy more if they have something 
to hide. 

In light of these issues, our goal is to construct mechanisms for privacy-aware agents, where privacy is 
accounted for the 'traditional' inputs to the mechanism (such as valuations, locations, etc.) but also, and 
for the first time to the best of our knowledge, with respect to the privacy valuation itself. 

The results of GR11| show, however, that this goal is too ambitious - no individually rational mechanism 
can compensate individuals for the information (dis)utility incurred due to information leaked about the 
privacy valuation from the public output unless the privacy valuations arc bounded. To overcome this 
obstacle we focus on mechanisms for large populations of agents: We propose a relaxation where loss in 
privacy is accounted for all agents whose valuations are bounded, where the bound increases as the agent 
population grows. Hence, in large enough populations truthfulness is provided for all (or most of) the agents. 
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For the small fractions of agents who value their privacy too much for the mechanism to compensate, we 
provide e-differential privacy with respect to whether their privacy valuations exceed the bound. The value 
of e improves (i.e., reduces) with the population size. 



1.1 Our Contributions 

Modeling The main contribution of this work is a new notion of privacy-aware mechanism design where 
wc examine critically previous modelings propose and justify a new model for privacy aware agents. 

We model privacy-aware agents to hold a 'traditional' game type and a privacy type, where for the latter 
agents need only have a conservative upper bound on how loss of privacy adversely affects their utility. Agents 
care about leakage of information on both the ir game and privacy types. These features are in an important 
difference with respect to previous work (e.g., |GR11 |) where a full characterization of the information utility 



was required to achieve truthfulness, and furthermore, mechanisms did not take into account the information 
cost of the privacy type. 

Note that if agents can have arbitrarily high privacy valuations, then it is impossible to a priori bound the 
information of a computation whose outcome depends on agents' private i nputs, or, alternatively, on their 



choice whether to participate or not (see also a more elaborate argument in [GRlll | in the specific context of 
mechanisms for selling private information for statistical computations). To sidestep this inherent difficulty, 
we opt for a lesser requirement from a privacy-aware mechanism: the mechanism should set a threshold on 
the privacy valuation v„iax f^nd a privacy parameter e such that the mechanism is incentive compatible w.r.t. 
agents whose privacy valuations are below Vmax and e-differential privacy is guaranteed for all agents. 

Construction of Privacy- Aware Mechanisms We next demonstrate that privacy-aware mechanisms 
are feasible. Our first result illustrates some of our techniques: in Section 231 we provide a simple privacy- 
aware poll between two or more alternatives. The main idea is to make (traditional) dis-utility due to 
mis-reporting dominate the information utility, and hence preserve truthfulness. We set a bound Vmax on 
the privacy valuations, and treat agents differently according to whether their valuations are above Vmax or 
not: for agents whose privacy valuations is below the bound, the mechanism ensures that the agents are 
provided with a fair reimbursement for their privacy loss. For agents whose privacy valuations are too high 
for the mechanism to compensate, we provide that their privacy valuations are protected in a e-differentially 
private way. As discussed above, this is in a sense the best we can hope to achieve. We then move our 
attention to large populations and we introduce the notion of admissible populations by making a somewhat 
mild assumption on the distribution of the valuations (i.e., finiteness of its moments). 

In Section [5] we present a generic c onstruction of privacy-aware mechanism. Our construction is based 



on the recent construction of |NST12| |. where we modify the mechanism and its analysis to accommodate 
privacy-agents sampled from an admissible population. We show that the mechanism achieves truthfulness 
for most agents and non-trivial accuracy. Finally in Section 15.21 we present a natural example of a privacy- 
aware mechanism that falls in our framework i.e., privacy-aware selling of digital goods. 

In a sense, our results show that when the outcome of a truthful (not necessarily privacy-aware) mech- 
anism is insensitive to each of its individual inputs (as is often the case when the underlying population 
is large), it is rational for most privacy-aware agents to report truthfully. This is because the information 
leaked about their private types is small, and hence bounded away from the decrease in utility that can be 
caused by misreporting their type. 

1.2 Other Related Work 

The cryptographic literature a lso inclu des references to "privacy preserving mechanism design" (an example 
is Naor, Pinkas and Sumner InpshI). We stress that our goals are different from these cryptographic 



realizations of mechanisms as in our setting the agents are worried about what the public outcome of a 
mechanism may leak about their types and privacy valuations, whereas the goal of cryptographic re alization s 
of mechanisms is to hide all information except for the outcome of the mechanism. As showed in MNT09{ . 



using cryptography to implement mechanism designs over an internet-like network is a non-trivial task, and 
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one needs to make sure that the properties of the mechanism (e.g., truthfulness) are preserved also by the 
cryptographic implementation of the mechanism. 

Independently from our work, Chen, Chong, Kash, Moran, and Vadhan (CCK+ll| also studied the 
problem of truthful mechanisms in the presence of agents that value privacy. The motivation for both their 
work and ours is similar, and in both the quantification o f privacy loss corresponds to the effect an agent's 
input has on the outcome of a mechanism. The model [CCK^ 111 present f or privacy-aware agents (and 
hence privacy-aware mechanisms) is different from ours in that in CCK+ll| agents are assumed to value 
privacy on a per-outcome basis, whereas our modeling utilizes a weaker assumption about the agents, i.e., 
that their privacy valuations depend on the overall (i.e., worst) outcome of the mechanism. Both modelings 
are well motivated, our reliance on a weaker assumption may lead to more robust mechanisms, where the 
per-outcome approach may lead to a richer set of privacy-aware mechanisms. 



2 Preliminaries 

We refer to discrete sets T and S as the type set, and the set of social alternatives respectively. For two 
vectors t, t' € T" we define the Hamming distance between t and t' as the number of entries on which t' 
differ, i.e., \{i : ti ^ t'^}\. Vectors that are within Hamming distance one are called neighboring. A mechanism 
M : T" — )- A(S') is a function that assigns for any vector of inputs t £ T" a distribution over S (the notation 
A (5*) denotes the set of probability distributions over the set S). The outcome of an execution of M on 
input t E T" is an element s £ S chosen according to the distribution M (T) . 

Definition 1 (Differential Privacy [dMNSO^). A mechanism M : T" — > A(5) preserves e- differential 
privacy if for all neighboring t,t' g T" and for all (measurable) subsets S' of S it holds that 

M{t){S') ^e" ■M{t'){S'). 

The following simple lemma follows directly from the above definition (the proofs for Lemma [T] and 
Theorem [T] below are not new and are included for completeness in Appendix E]) : 

Lemma 1. Let M : T" — > A(S') be a mechanism that preserves e- differential privacy and let g : S ^ M-". 
Then, for all neighboring t, t' G T" 

^s~M{t)[g{s)] < e=E,^M(t')[f('S)]- 
In particular, if e < 1 and g : S ^ [0, 1], 

|E^~A/(i)[.9('S)] - E^~M(t')[3(s)]| < 2e- 

A simple corollary of Lemma [T] is that jE^^^/i-f) [g(s)] — E^^tv/^j/-) [g(s)] | < 4e for neighboring t,t' and 
g:S^ hl,l]- 

Definition 2 f jMTOTt ). Let f : S x T"^ ^ M^" and let e > 0. The exponential mechanism for f with 
parameter e is 

Theorem 1 ([MTo3). LetAf be the maximum over all s G S and neighboring t, t' g T" of /(s, t) — f{s, t'). 

£ 

Mj^' preserves e- differential privacy. 

Definition 3 (Mutual Information). Let X,Y be two random variables. The mutual information between 
X and Y is defined as 

I{X- Y) = H{X) + H{Y) - H{X, Y), 
where H{X) = — X^i^gs P'"!^ ~ ' l^S (Pr[A" = x\) is the Shannon entropy of X . 
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It is well known that I{X; Y) = H{X) — H{X\Y), i.e. I{X; Y) measures the reduction in entropy in X 
caused by conditioning on Y (and symmetrically, I{X;Y) = I{Y;X) = H(Y) — H(Y\X)). The following 
simple observation follows from the data processing inequality (see, e.g., |CT91 . pp. 32]): 

Observation 1. For all (randomized) functions f, I{f{X);Y) < I{X,Y). 

Observation 2. Let M : T" — > A(S') be an s differentially private mechanism, then for all random variables 
X = {Xi, . . .,Xn) e A(r") it holds that J(Xi; M(X), < e. 



3 Quantifying Information Utility 

Our model is similar to the standard model of mechanism design, with the difference that agents participating 
in the execution of a mechanism care about their privacy. In the standard model, an agent's type ti expresses 
quantities such as a valuation of a good for sale, location, etc., the mechanism chooses an alternative s, and 
the agent's utility is a function of ti and s (and sometimes, monetary transfers). 

When considering privacy-aware agents, we need to introduce the information utility into their utility 
functions. A first issue that emerges is how should this dis-utility be quantified? Note that as different agents 
may value privacy differently, the quantification should be parametrized by agents' privacy preferences. We 
denote by w,; the privacy preference of agent i. That is, an agent type is now composed of the 'traditional' 
type ti, and a privacy preference Vi. A second issue that now emerges is that the alternative chosen by the 
mechanism can leak information about both ti and Vi, and hence leakage about Vi needs also be taken into 
account. 



How is information utility quantified in prior work In an early work, McGrew, Porter, and S hoham 
jMPS03 | introduced privacy into agents' utility in the context of non-cooperative computing (NCC) jSTOSl ]. 



In their model, agents only care about the case where other agents learn their private types with certainty. 
This means that privacy is either completely preserved or completely breached, and hence information utility 
is quantified to be either zero (no breach) or an agent dependent value Vi > 0. As it is often the case that 
leaked information is partial or uncertain, we are interested in more refined measures that take partial 
exposure into account. 

A recent work by Ghosh and Roth jGRll | considers a setting where a data analyst wishes to perform 



a computation that preserves e-differential privacy and compensates participating agents for their privacy 
loss. They assume a model where each agent's dis-utility is proportional to the privacy parameter e. I.e., 
the ith agent's dis-utility is 

u™^ = Vi- e, 

where > is part of the agent's private type. A problem with this quantification is that while e measures 
the worst effect the e-differenti ally priv ate computation can have on privacy, the typical effect on agent i 
can be significantly lower (see [PRVIOl ]). Furthermore, it can depend on the other agents' inputs to the 
computation. Another problem, that will be further discussed later, is that this quantification does not 
consider the information utility due to leakage of information about Vi itself; 



The third example we are aware of is from another recent work, by Xiao |Xiall| . Similarly to the present 
work, Xiao considers the setting of mechanism design with privacy-aware agents. The information utility is 
modeled to be 

uf"^ = v,- I{t,;M{t^„a{U))), 

where > is the agent privacy valuation. Note that with this measure, the dis-utility of agent i depends 
on the distribution of her and the other agents' types, and on her own strategy a. The following example 
demonstrates that this dependency on a is problematic. 

Consider the single-agent mechanism below, where the agent's private type consists of a single bit: 

Example 1 (The "Rye or Wholewheat" game). Alice is preparing a sandwich for Bob and inquires whether 
he prefers Rye (R) or Wholewheat (W). Bob wants to enjoy his favorite sandwich, but does not want Alice 
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to learn his preference. Assume that Bob's type is uniformly chosen in {R, W} and consider these two 
possibilities for Bob's strategy: 

1. If Bob provides his true preference he will enjoy the sandwich. However, his information (dis)utility 
would be maximized as I(tBob] M{atruthfui{tBob))) = 1- 

2. If Bob answers at random he will enjoy the sandwich with probability one-halj^ However, as his 
response does not depend on his preference no loss in privacy would be incurred, hence we get 

om [tBob))) = . 

Note that since Bob's type is R or W with equal probabihty Ahce's views of Bob's actions (and hence 
also the outcome of the mechanism) are distributed identically whether he uses a truthful or cfrandom- Hence, 
while mutual information differs dramatically between the two strategies - suggesting that Bob is suffering a 
privacy loss due to Alice learning his type in one but not in the other - it is impossible for Alice to distinguish 
between the two cases! 

A few more words are in place regarding the source of this problem. First note that while the example 
demonstrates that I{tBob] M{cr{tBob))) is a problematic as a measure of privacy it does not imply that 
'^random (uor the One time pad) is at fault (in fact, arandom provides Bob with perfect privacy even in a 
setting where Alice gets to know which strategy Bob uses, a guarantee atruthfui definitely does not provide). 
What the example capitalizes on is the fact that the standard game-theoretic modeling docs not rule out 
the possibility that Alice does not get to see what Bob's strategy is. In such situations, it can happen that 
the more robust CFrandom is an overkill, as it provides Bob with less utility. We hence argue that the notion 
of information cost should be free of making assumptions on Alice's knowledge of a. 



3.1 Our Approach 

We deviate from the works cited above as we do not present a new measure for information utility. We use 
a significantly weaker notion instea d. To m otivate our approach, re-consider the measures discussed above. 
Looking first at the measure in [GRII . i.e., Vi ■ s, we note that while in e-diffcrential mechanisms the 



ratio Pr[M(t) = s]/ Pr[M{t') = s] is bounded by for all neighboring t,t' and s, it is plausible that the 
worst case behavior (i.e., outputting s such that Pr[M(t) = s]/Pr[M(i') = s] = e^) occurs with only a tiny 
probability. This suggest that while Vi ■ e may not be a good mea sure fo r information utility, it can serve 
as a good upper bound for this utility. Examining the measure in Xiall| and trying to avoid the problem 



demonstrated in Example [T] above, we note that by Observation [T] /(t^; M(i)) > I{ti; M{t-i, a{ti))) for all 
(T, hence, we get that Vi ■ I{ti \ M{t)) is another plausible upper bound for information utility. Finally, taking 
into account Observation [2] we get that I{ti] M{t)) < e and hence we choose to use Vi • e as it is the weaker 
of these bounds. 

Note 1. We emphasize that although our usage of the term u,; • e is .syntactically similar to that of 
our usage of this quantity is conceptually very different. In particular, while loss of pri vacy ca nnot be used 
in our constructions for deterring non-truthful agents, the constructions (and proofs) in fGRll l use the fact 
that the information utility is (at least) Vi ■ s for arguing truthfulness. 

Note 2. Lemma{^ .supports using Vi ■ £ as an upperbound for information utility in the follwing sense. An 
individual's concern about her privacy corresponds to a potential decrease in future utility due to information 
learned about her. an upper bound on information utility hence should correspond to this (potential) loss in 
future utility. By Lemma\^ the information contributed by individual i affects the expectation of every non- 
negative (similarly, non-positive) function g by at most a factor of . Let Gi : S* — > K describe how the 
future utility of individual i depends on the outcome of M . By Lemma [H the information utility of that 
individual is bounded by 

max(e^ - 1) • E^^M(t) \Gi{s)\ « e • maxE^^M(t) |G,;(s)| , 



This is equivalent to encrypting Bob's type using a one time pad. 



6 



where the approximation holds for small e. See also a related discussion in 



Privacy of Vi The mechanisms presented in jGRlll | for selhng private information do not protect the 
privacy of Vi nor they account for the information (dis)utihty generated by the leakage of Vi. It is further 
shown that with unbounded ViS it is impossible to construct mechanisms that compensate agents for their 
loss in privacy and achieve reasonable accuracy (in the sense that enough agents sell their information). 

Our mechanisms provide an intermediate solution. First, we provide e-differential privacy to all agents, 
where the guarantee is with respect to their combined type, i.e., (ti,Vi), and where e decreases with the 
number of agents n. This means that privacy improves as n grows. 

Furthermore our constructions guarantee that truthfulness is dominant - taking information utility about 
the combined type (ti,Vi) into account - for all agents for which Vi < Wmaxi where under a very mild 
assumption on the distribution of Vi the bound Vmax grows with n and the fraction of agents for which 
Vi > Vmax decrcascs with n. 



4 The Model 

The Mechanism Let 5 be a finite set of alternatives (a.k.a. social alternatives), let T be a finite type 
set and consider a set of n agents. We consider direct revelation mechanisms that given the declaration of 
agents about their types selects a social alternative s € 5 and makes s public. To isolate loss of privacy 
due to publication of s from other potential sources of leakage, we will assume that every other information 
(including, e.g., the agents' declared types and individual monetary transfers) is completely hidden using 
cryptographic or other techniques. 



The Objective Function The goal of the designer is defined via a real, non-negative objective function 
over the true types of the agents, f{t, s) that needs to be optimized (by choosing s). 

/ : T" X 5^ [0,nA/]. 



Following [DMNSOd |MT07| we define the sensitivity of / to be 

A/ = max |/(f,s) -/(£', s)| 

where the maximum is taken over all neighboring t, r G T" and s S. We assume that for all s the minimum 
value of f{t, s) is and then, given that sensitivity is A/ by a hybrid argument we get that / < nAf. 



Privacy- Avifare Agents We extend the traditional setting of selfish agents to include agents who care not 
only about their utility uf^^ from the outcome s of the mechanism, but also about the (negative) information 
utility uf^^ incurred from the leakage of information about their private type through the public output s. 
For simplicity, we consider a setting where the overall utility of an agent is the sum of the twoj^ 

An agent's type is modeled by a pair — {ti, Vi) E T x where T is the "traditional" game type 
and Vi is the privacy valuation of agent i. We emphasize that agents care about the privacy of the whole 
pair and the information utility corresponds to the loss in privacy of both ti and Vi (hence, one cannot 
simply publish Vi). The vectors t = {ti, . . . , i„) and v ~ (vi, . . . , u„) denote the types of all agents. Trying 
to maximize her utility, agent i may hence act strategically and declare t- — <Ti{Ti) — {t^,v'j) to M instead 

of Ti. 



Admittedly, this separation of the utility function is sometimes artificial. However, we find it conceptually helpful. 
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The "traditional" game utility of agent i is defined as uf : T x S ^ [—1,1]. Following our discussion 
above, we define uf^^ : E-° M-°, and the only assumption we make is that 

for e being the parameter of the differentially private mechanism executed (i.e., e'^ = max {A{{t){S)/M{t')(S)) 
where the maximum is taken over all neig hboring t,t' G T" and S' C S). Note that unlike uf^^ that only 
depends on the outcome of the mechanism, u^^^ depends on the mechanism itself. 

In our analysis we identify a subset of agents that we call participating for whom truthtelling is strictly 
dominant. A mechanism approximately implements / if assuming that participating agents act truthfully 
(and other agents act arbitrarily) it outputs s that approximately optimizes /. 



4.1 Warmup: A Privacy- Aware Poll 

The following simple electronic poll will serve to illustrate some of our ideas: 

Example 2 (An Electronic Poll). An electronic publisher wishes to determine which of its m>2 electronic 
magazines is more popular. Every agent is asked to specify her favorite magazine, i.e., ti £ [m], and will 
receive in exchange an electronic copy of it. For simplicity, we assume that agents ' utility does not depend 
on the poll outcome. 

Following our convention, we assume ideal cryptography here, that is, no information beyond the outcome 
of the poll is leaked. In particular, every agent receives the electronic magazine without anybody ( including 
the publisher) knowing which magazine has been transferred. Agents, however, are privacy- aware, and hence 
take into account that the outcome of the poll itself reveals information about their preferences. 

Denote by t' the vector of agents' declarations. For s G [m] let f{s,t') = \{i\t'^ = s}\ and note that 
A/ = 1. Consider the exponential mechanism M = as in Definition [2] I.e., 

Pr[A/(i') = j] = 



„en'/2 ' 



where n'j = f{j,t') is the number of agents who declared they rank magazine j first. By Theorem [1] M 
preserves £-differential privacy. 
Note that if n'^ > n'^ + k then 

Pr[Af (f) = < —— < ^T-TTTJ = e-^'/'. 

Hence, 

Pr[M(i') outputs i such that n'^ < max?^' - k] < (m - 1)6^"''/^ 
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The agent utilities are uf^^ — uf^^ where 



• uf^^^ is the utility that the agent gains from receiving the magazine she specified she prefers. Note that 
this utility depends only on the declared type and it is maximized for ti, the true type of the agent; 
we assume that uf^^{ti) — uf^^{t/j) > g (Alternatively, the publisher does not care if agent i reports t'^ 

if uf^Hu) - uf^'Ht'i) < 9)- 

• u\ < e ■ Vi is the privacy loss from the mechanism. 

Note that e < g/vi suffices for making agent i truthful: acting untruthfully agent i will lose at least g in 
uf^^ and gain no more than svi in uj'^^. Denote by Uj the number of agents who rank magazine j first 
(note the difference from n'^ that correspond to declared types). To demonstrate that the mechanism is 
efficient, we need to make some (hopefully reasonable) assumptions on the distribution of Vi. We explore 
three possibilities: 
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Bounded Vi We begin with a simplified setting where we assume that there exists Vmax = 0(1) such that 
Wi : Vi < Vmax- In this case it is enough to set e < g/vmax = to make truthfulness dominant for all 

agents. Hence, assuming all agents are truthful, we get n'j = rij for all j G [to] and hence the probability 
that M{t') ~ M{t) outputs I such that ni < maxj rij — k is bounded by (to — l)e~^'^/^. 

Note that in this case the computation output leaks no information about the privacy valuations v. 



Bounded Vi, Except for a Small Number of Agents A more realistic setting allows for a small 
number of agents with Vi > Vmax- We change the mechanism M to also consider the reported v'^ so that 
inputs from agents with v[ > Vmax are ignored. Regardless of what agents with vt > Vmax report, we call 
them non-participating. 

As before, by setting e < g/vmax we make truthfulness dominant for all agents with Vi < Vmax- We can 
hence guarantee a non-trivial accuracy. Let n„p be the number of non-participating agents. In the worst 
case, non-participating agents deflate the count of a popular magazine and inflate the count of an unpopular 
magazine, making it look more popular than it really is. Taking this into account, we get that 

Pr[A/(t') outputs £ such that ni < maxn^ — fc — 2n„p] < (m — l)e~'^'^/^. 

j 

Note that we lose truthfulness for non-participating agents. We do, however, guarantee e-differential 
privacy for these agents. 



Large Populations Assume we do not care if the mechanism does not output the most popular choice if 
it does not have signiflcant advantage over the other, e.g., when k + n„p = 0(n") for some < a < 1. This 
allows us to set e(n) = n~" and hence truthfulness is dominant for agents with Vi < g/s = Vmaxin) € 0{n'^). 
Note that Vmax grows with n, hence we expect the fraction of non-participants Unp/n to diminish with n. If 
n is large enough so that the fraction of agents for which Vi > Vmaxin) is at most l/n^~" then we get the 
desired accuracy. 

As before, we lose truthfulness for non-participating agents, and only guarantee e(n)-differential privacy 
for the non-participating agents. Note, however, that the fraction of non-participating agents diminishes 
with n, and, furthermore, their privacy guarantee improves with n (i.e., s{n) decreases). 



4.2 Admissible Privacy Valuations 

In the rest of the paper we only focus on large populations (the analysis can be easily modified for the 
case where Vi is bounded except for a small number of agents). We will design our mechanisms for "nicely- 
behaving" populations: 

Definition 4 (Admissible Valuations). A population of n agents is said to have (a, /3) -admissible valuations 

|{^ : v^ > ^ 
n ~ 

To partly justify our focus on admissible valuations, consider the case where Vi are chosen, i.i.d., from 
some underlying distribution T) over M-''. 

Definition 5 (Admissible Valuation Distribution). A valuation distribution T) is called (a, /3) -admissible if 

Pr [i; > n"] = 0{n-'^). 

Note that if V has finite expectation, then (using Markov's inequality) Pr[v > n"] < E[f]/n" ~ 0{n~"), 
and hence 2? is (a, /3)-admissible for all P < a. liT) has finite variance then (using Chebyshev's inequality) 
Pr[?; > n"] < Var[w]/(n" - ^[v]Y = 0(71"^"), and hence V is (a, /3)-admissible for all /3 < 2a. More 
generally, consider the following simple generalization of Chebyshev's inequality to even pth moment: 

Pr[|A - E[A]| > t] = Pr[(A - E[A])p > t^] < — ^ E[A])p] 



tP 
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Using this inequality in the argument above we get that if T> has finite even pth moment then 23 is (a, /?)- 
admissible for all /? < pa. We conclude that if T) has finite pth moment then T) is (a, 1 — a)-admissible for 
ot > + !)■ In particular, if V has finite moments of all orders then 23 is (a, 1 — a)-admissible for all 
a e (0,1). 

We can even consider a notion of strong admissibility: 

Definition 6 (Strongly Admissible Valuation Distribution). A valuation distribution 23 is called a-strongly 
admissible if 

Pr [v > (logn)"] = 
where n^'^^^^ denotes a function that is negligible in n. 



.(1) 



For example, the Normal distribution is a-strongly admissible. In our analysis, however, we only use the 
more conservative notion of admissibility as in definitions |4l [5] 

These simple observations suggest that (a, 1 — a)-admissibility is a relatively mild assumption that would 
typically hold in large populations even for small values of a. 

4.3 The Privacy- Aware Poll with Admissible Valuations 

Returning to our example, let a be the smallest positive value such that the agent population can be assumed 
to be (a, 1 — a)-admissiblc. By setting Vmax = a-nd e — g/vmax — 5"- " we get that n^p < n-n~^^~°'^ = n". 
Finally, setting k ~ n"(logri)^ logm/g we get the following: 

Claim 1. The probability that M{t') outputs i such that ng < maxj Uj — 2k is negligible in n. 

5 A Generic Construction of Privacy- Aware Mechanisms 

We now present a gener ic feasibility result for privacy-aware mechanisms. Our construction is based on the 



construction of |NST12l | , where differential privacy is used as a tool for mechanism design. The hope is that 
existence of this generic construction, a relatively simple modification of ,NST12j , is a signal that our model 
of privacy-aware mechanisms allows constructing mechanisms for many other tasks. 



Reactions We first change our model to incorporate the notion of reactions introduced in NST12j p| 
Traditionally, an agent's utility is a function of her private type an d the so cial alternative, and the issue 
of how agents exploit the social choice is not treated explicitly. In NST12l | this choice was made explicit 



such that after a social choice is made agen ts need t o take an action (denoted reaction) to exploit the social 
alternative and determine their utility. In NST12l | (and likewise in this work) allowing the mechanism to 



sometimes restrict the reactions of agents serves as a deterrent against non-truthful agents. 

Let i? be a finite set of reactions. We modify the definition of the utility from the outcome of the 
mechanism to 

uf'^* -.TxSxR^ [-1,1]. 

Given ti, s define 

ri{t^, s) = argmax^gfl(Mf^*(ii, s, r)) 

to be the optimal reaction for agent i on outcome s. 

To illustrate the concept of reactions, consider a mechanism for setting a price for a unlimited supply 
good (such as in Example [3] appearing below). Once the mechanism chooses a price s the possible reactions 
are buy (i.e., pay s and get the good) and not buy (i.e., do not pay s and do not get the good), and reactions 
are kept hidden by assuming payment and reception of the digital good using perfect cryptography. In this 
example agents reactions may be restricted to buy whenever they bid at least the selected price s, and not 
buy otherwise. 



^ While the standard game-t heoretic modeling docs not explicitly include reactions, in many settings their introduction is 
natural. We refer the reader to |NST12| for further discussion of this change in the standard model. 
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Utility Gap We assume the existence of a positive gap g such that for aU ti ^ t'^ there exist s for which 
the optimal reactions are distinct, and, furthermore, uf^^{ti, s,ri{ti, s)) > uf^^{ti, s,ri{t^, s)) + g. In many 
setting, a gap g can be created by considering a discrete set of social choices. As in our polling example, an 
alternative interpretation of the gap g may be that the mechanism designer not care if agent i reports t'^ if 
uf'^^iU, s, r,{U, s)) < uf^^iU, s, r,{t[, s)) + g. 

5.1 The Construction 

Given a finite type set T, a finite set S of alternatives and an objective function / : T" x — > R with 
sensitivity A/, we construct a mechanism for approximately implementing /. 

Let n be the number of agents, and let a be the smallest positive value such that the agent population 
can be assumed to be (a, 1 — Q;)-admissiblc. Let Vmax = . The participating agents will be those with 
privacy valuations lower than Vmax- Choose tj^ G T to be an arbitrary element of T. Non-participating 
agents wih be asked to declare t_\_. 

Let i5 G [0, 1], e > be parameters to be set later. Agents are asked to declare ti if Vi < v,nax and t± 
otherwise. Let t[ be the declaration of agent i. On input <' = t'j^, . . . , the mechanism executes as follows: 

ALGORITHM 1: The generic mechanism M. 

Input: A vector of types t' £ T". 
Output: A social choice s £ S. 

M executes Mi with probability 1 — 5 and A'h otlierwise, where Mi, M2 axe as follows: 

Mechanism Mi For all s £ S* and t' £ T" , choose s £ S according to the exponential mechanism (t'). 
Mechanism Af2 Choose s £ S uniformly at random. 

The mechanism M also restrict all agents to their optimal reactions according to their declarations, i.e., ri(t^,s)|3 

We begin by analyzing for which agents truthtelling is a dominant strategy: 
Claim 2. // {v„iax + 4)e < then truthtelling is dominant for all agents with Vi < Vmax- 
Proof. We first analyze the effect of misreporting in Mi and A/2: 

Misreporting in A/i As uf^^(ti,s,r) e [—1, 1] we can use the simple corollary following Lemma [1] and 
get that for all possible declarations of the other agents t'_i and all t[: 

where the first inequality follows from u^^^iti, s, ri(i-, s)) < uf^^{ti,s, ri{ti,s)). In words, misreporting can 
gain at most 4e in the expected wf^^H Noting that misreporting can gain agent i at most Vi ■ e in uf^^, we 
get that the total gain in utility due to misreporting by agents with Vi < Vmax is {Vmax + 4)£. 

Misreporting in A/2 If t'^ ^ ti then with probability at least ^ we get that ri(t\, s) 7^ r-,i(<i, s). Since the 
mechanism restricts agent I's reaction to ^^(i^, s) we get that 

Es^M2(t'_.,ti)[uf^*(^4,s,?'»(ij,s))] - ^!>~M^{t'_^X.)\u^^^(t^,s,ri(t!^,s))\ > ||^, 
''We note that for the analysis it suffices to restrict reactions only when A/2 is activated. 

^Similarly, even if reactions are not restricted when is activated we get that: ^sr^Mi(t' . ,t' ) rj (ij, s))] — 

■Es~A/i{t' . ["f'^^C**! •'))] < 4£. We only need reactions to be restricted when A/2 is activated. 
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where g is the minimal utiUty gap due to not acting according to the optimal reaction. Note that, as M2 
ignores its input, misreporting does not yield a change in u™^. We get that the total loss in utility in M2 
due to misreporting is at least g/IS*!. 

We get that if {v„iax +4)e < (5y|j then overall gain in utility due to misreporting is negative for all agents 
with Vi < Vmax, hcucc truthtelling is dominant for these agents. □ 

Let opt(t) = maxsg5 f{t, s) be the optimal value for /. We next show that our mechanism approximately 
recovers opt(t). 

Claim 3. // {vmax + 4)e < then 

E.^A/(t')][/(^' ^)] > opt(0 - A/ • {Sn + + 2 ln(n|5|)/e) . 

Proof. Define opt' = niaxj f{t', s). Denote by tk the vector constructed from the k first entries of t and the 
n — k last entries of t' . For all s we have that 

n-l 

/(f, s) = /(fo, s) = J2 ifitk, s) - f{h+i,s)) + fit, s). 

Note that by Claim[2]t'; ^ ti for at most entries, and hence f{tk, s) — f{tk+i, s) 7^ for at most n" values 
of k, in which case it is upper bounded by A/. We get hence that opt' > opt(<) — n"A/. 
We get that Mi{t') outputs s' such that f{t',s') < opt' - 2A/ln(ri,|5'|)/e with probability 

exp(e/(t',s')/2A/) ^ exp(e(opt' - 2A/ ln(n|^|)/£)/2A/) _ 1 



Esesexp(e/(i',s)/2A/) - cxp(eopt'/2A/) 

Using the union bound (over elements of 5), and the fact that opt' < nAf, we get a lower bound on the 
expected revenue of Mi as follows: 

(to > (opt'-2A/ln(n|5|)/£) 

> opt'-2A/ln(n|S'|)/£- A/ 

> opt(t) - 2n"A/ - 2A/ln(n|S'|)/e. 



We conclude that 



> (1-5) (opt(t) - 2ri"A/ - 2A/ln(n|5|)/e) 

> opt(t) - SnAf-2n"Af - 2A/ln(?i|S'|)/e. 



Setting e = „-(i+")/2^g ln(n|5|)/|5| and 6 = 2n("-i)/ Vl-S"! ln(n|5|)/g we get 



□ 



Theorem 2. Let n be the number of agents, T be a finite type set and S a finite set of alternatives. Let 
f : T" X S* — > R &e an objective function with sensitivity Af and M be the mechanism described in Algorithm]^ 
If a is such that the agent population can be assumed to be (a, 1 ~ a) -admissible, then M recovers opt(t) 

to within additive difference 0/ O (A/n(i+")/V|5'| ln(n|S'|)/.g) . 

The relative accuracy of our mechanisms, in the sense of the difference between the optimal value when 
agents are privacy-aware or not, increases with larger populations. As described before, natural distributions 
of the privacy valuations will be (a, 1 — a)-admissible even for very small values of a, and therefore the 
dominating term in the expression in Theorem [2] can be made arbitrarily close to 0{y/n). 
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5.2 Example: Privacy- Aware Selling of Digital Goods 

We describe now an example of a natural privacy-aware mechanism that naturally falls within our framework. 

Example 3 (Pricing a Digital Good). An auctioneer selling a digital good wishes to design a single price 
mechanism that would (approximately) optimize her revenue. Every party has a valuation ti € Q = 
{0, |, |, . . . , 1} for the good (for some constant q), and a privacy preference Vi. Agents are asked to de- 
clare Ti — {ti,Vi) to the mechanism, which chooses a price p for the good. Denote by r- — the actual 
declaration of agent i. If t[ > p then agent i receives the good and pays p, otherwise, agent i learns p but 
does not pay nor receive the good. Agents prefer receiving the good to not receiving it. 

• The utility u^^^ is the 'traditional' utility, i.e., zero if agent i does not receive the good, and P+ ^ 
otherwise, where the additive is used for modeling preference to receive the good. 

• For u^^^ , we assume that whether agent i received the good and paid for it can be kept completely hidden 
from all other parties (this can be implemented using cryptographic techniques). Hence, only leakage 

r 

due to making p public affects u . 

Consider now the auctioneer from Example [31 and assume that the valuations ti are taken from T = 
{0, i, |, . . . , 1} and similarly, the price p G S = |, . . . , 1} for some integer constant g > 1. Let a be the 
smallest value such that the agent population can be assumed to be (a, 1 — Q;)-admissible. 

Defining the reactions to be {buy, not buy} and optimal reactions ri{t,p) = buy \it> p and not buy 
otherwise we get that the gap g is l/2q. 

Suppose the designer goal is to recover the optimal revenue, i.e., va&yip^g f{t,p) where f{t,p) — p ■ \{i : 
ti > p}\ and note that A/ = 1. 

Using Theorem [5] we get a privacy-aware mechanism that recovers the optimal revenue to within additive 
difference of O (^Afn'^^+°'')/^y/\S\ln{n\S\)/g'^ = O (n(i+")/2g^ln(ri9)) . 

Note that the accuracy of this privacy aware mechani sm is only (essentially) a factor 0{n'^) away from 



the similar (non-privacy aware) mechanism from NST12 [ 
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A Omitted Proofs 

A.l Proof of Lemma [1] 

Proof. Let t,t' ,g be as in the lemma. 

^sr.M{tMs)] =Y,M{t){s)g{s) <Y,e'Mit')is)g{s) = e' ■ E,^M(t')[5(s)], 

ses ses 

where the inequality follows since M provides e-differential privacy, and g is non-negative. For e < 1 and 
g : S ^ [0,1] we get 

Es~A/(t)[ff(s)] - E,^M(t')[5(s)] < (e" - 1) ■ E,^M(t')[5(s)] <e'' ~1, 

where the last inequality holds because g returns a values in [0,1]. Similarly, we get 'Eisr^M{t')[gis)] — 
E,s^A/(t)[,9(s)] hence 

|E,^A/(t)[g(s)] -E,^M(t')[5(s)]| <e/ -l<2e, 
where the last inequality follows noting that {e^ — 1) < 2e for < e < 1- CH 
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A. 2 Proof of Theorem [T] 

Proof. Let t, t' be neighboring, and 5" C 5". 



^ ^^^^ " h,T..es^M^f{s',t)) 



E 



seS' 



cxp(^(/(g,t) - /(g,tO))exp(^/(.,tO) 
exp( i) - /(s', i'))) exp( t'))) 



exp(§)exp(2fj/(s,t')) 

^ E 



J^, Es'Gsexp(-f)exp(2f^/(s',t'))) 
= exp(e)M|(t)(^'), 

where the inequality follows by recalling that A/ > \f{s,t) — f{s,t')\ for all s,t,t'. □ 
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